

<!DOCTYPE html>
<!--[if IE 8]><html class="no-js lt-ie9" lang="en" > <![endif]-->
<!--[if gt IE 8]><!--> <html class="no-js" lang="en" > <!--<![endif]-->
<head>
  <meta charset="utf-8">
  
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  
  <title>4. Tutorial &mdash; UCTB  documentation</title>
  

  
  

  

  
  
    

  

  
  
    <link rel="stylesheet" href="../_static/css/theme.css" type="text/css" />
  

  

  
        <link rel="index" title="Index"
              href="../genindex.html"/>
        <link rel="search" title="Search" href="../search.html"/>
    <link rel="top" title="UCTB  documentation" href="../index.html"/>
        <link rel="next" title="5. API Reference" href="../APIReference.html"/>
        <link rel="prev" title="3. Quick start" href="quickstart.html"/> 

  
  <script src="../_static/js/modernizr.min.js"></script>

</head>

<body class="wy-body-for-nav" role="document">

   
  <div class="wy-grid-for-nav">

    
    <nav data-toggle="wy-nav-shift" class="wy-nav-side">
      <div class="wy-side-scroll">
        <div class="wy-side-nav-search">
          

          
            <a href="../index.html" class="icon icon-home"> UCTB
          

          
          </a>

          
            
            
          

          
<div role="search">
  <form id="rtd-search-form" class="wy-form" action="../search.html" method="get">
    <input type="text" name="q" placeholder="Search docs" />
    <input type="hidden" name="check_keywords" value="yes" />
    <input type="hidden" name="area" value="default" />
  </form>
</div>

          
        </div>

        <div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
          
            
            
                <ul class="current">
<li class="toctree-l1"><a class="reference internal" href="introduction.html">1. Introduction</a></li>
<li class="toctree-l1"><a class="reference internal" href="installation.html">2. Installation</a></li>
<li class="toctree-l1"><a class="reference internal" href="quickstart.html">3. Quick start</a></li>
<li class="toctree-l1 current"><a class="current reference internal" href="#">4. Tutorial</a><ul>
<li class="toctree-l2"><a class="reference internal" href="#use-datasets-from-uctb">4.1. Use datasets from UCTB</a></li>
<li class="toctree-l2"><a class="reference internal" href="#build-your-own-datasets">4.2. Build your own datasets</a></li>
<li class="toctree-l2"><a class="reference internal" href="#use-build-in-models-from-uctb">4.3. Use build-in models from UCTB</a><ul>
<li class="toctree-l3"><a class="reference internal" href="#use-single-temporal-feature-in-regression">4.3.1. Use single temporal feature in regression</a></li>
<li class="toctree-l3"><a class="reference internal" href="#use-multiple-temporal-features-in-regression">4.3.2. Use multiple temporal features in regression</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="#build-your-own-model-using-uctb">4.4. Build your own model using UCTB</a></li>
<li class="toctree-l2"><a class="reference internal" href="#build-your-own-graph-with-stmeta">4.5. Build your own graph with STMeta</a><ul>
<li class="toctree-l3"><a class="reference internal" href="#top-k-graph">4.5.1. Top-K graph</a></li>
<li class="toctree-l3"><a class="reference internal" href="#realize-topk-graph-analysis-module">4.5.2. Realize TopK graph analysis module</a></li>
<li class="toctree-l3"><a class="reference internal" href="#redefine-the-call-statement-of-the-above-class">4.5.3. Redefine the call statement of the above class</a></li>
<li class="toctree-l3"><a class="reference internal" href="#modify-the-function-call-location">4.5.4. Modify the function call location</a></li>
</ul>
</li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../APIReference.html">5. API Reference</a></li>
<li class="toctree-l1"><a class="reference internal" href="all_results.html">6. Results on different datasets</a></li>
<li class="toctree-l1"><a class="reference internal" href="uctb_group.html">7. About us (UCTB Group)</a></li>
</ul>

            
          
        </div>
      </div>
    </nav>

    <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">

      
      <nav class="wy-nav-top" role="navigation" aria-label="top navigation">
        
          <i data-toggle="wy-nav-top" class="fa fa-bars"></i>
          <a href="../index.html">UCTB</a>
        
      </nav>


      
      <div class="wy-nav-content">
        <div class="rst-content">
          













<div role="navigation" aria-label="breadcrumbs navigation">
  <ul class="wy-breadcrumbs">
    
      <li><a href="../index.html">Docs</a> &raquo;</li>
        
      <li>4. Tutorial</li>
    
    
      <li class="wy-breadcrumbs-aside">
        
            
            <a href="../_sources/md_file/tutorial.md.txt" rel="nofollow"> View page source</a>
          
        
      </li>
    
  </ul>
  <hr/>
</div>
          <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
           <div itemprop="articleBody">
            
  <div class="section" id="tutorial">
<h1>4. Tutorial<a class="headerlink" href="#tutorial" title="Permalink to this headline">¶</a></h1>
<div class="section" id="use-datasets-from-uctb">
<h2>4.1. Use datasets from UCTB<a class="headerlink" href="#use-datasets-from-uctb" title="Permalink to this headline">¶</a></h2>
<p>UCTB is designed for urban computing in various scenarios. Currently, It presets a public dataset about bikesharing. This dataset was collected from U.S. open data portals, including 49 million, 13 million, and 14 million historical flow records in <a class="reference external" href="https://www.citibikenyc.com/system-data">New York City</a> (<code class="docutils literal notranslate"><span class="pre">NYC</span></code>), <a class="reference external" href="https://www.divvybikes.com/system-data">Chicago</a> (<code class="docutils literal notranslate"><span class="pre">Chicago</span></code>) and <a class="reference external" href="https://www.capitalbikeshare.com/system-data">Washington, D.C</a> (<code class="docutils literal notranslate"><span class="pre">DC</span></code>), respectively. Each record contains the start station, start time, stop station, stop time, etc. We predict the number of bikesharing demands in each station (i.e., the number of bike borrowers).</p>
<p>In the future version, we consider releasing more datasets covering other applications such as ridesharing, metro traffic flow, and electrical charging station usage. <strong>If you are interested in this project, making a contribution to the dataset is strongly welcomed :)</strong></p>
<p>To help better accuse dataset, UCTB provides data loader APIs <code class="docutils literal notranslate"><span class="pre">UCTB.dataset.data_loader</span></code>, which can be used to preprocess data, including data division, normalization, and extract temporal and spatial knowledge.</p>
<p>In the following tutorial, we illustrate how to use <code class="docutils literal notranslate"><span class="pre">UCTB.dataset.data_loader</span></code> APIs to inspect the bikesharing dataset.</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">UCTB.dataset.data_loader</span> <span class="kn">import</span> <span class="n">NodeTrafficLoader</span>
</pre></div>
</div>
<p>We use 10% (<code class="docutils literal notranslate"><span class="pre">data_range=0.1</span></code>) of the bike data in New York as an example. Firstly, let’s initialize a <code class="docutils literal notranslate"><span class="pre">NodeTrafficLoader</span></code> object:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">data_loader</span> <span class="o">=</span> <span class="n">NodeTrafficLoader</span><span class="p">(</span><span class="n">data_range</span><span class="o">=</span><span class="mf">0.1</span><span class="p">,</span> <span class="n">dataset</span><span class="o">=</span><span class="s1">&#39;Bike&#39;</span><span class="p">,</span> <span class="n">city</span><span class="o">=</span><span class="s1">&#39;NYC&#39;</span><span class="p">)</span>
</pre></div>
</div>
<p>Take a look at the necessary information about the dataset:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="c1"># Traffic data </span>
<span class="nb">print</span><span class="p">(</span><span class="s1">&#39;Data time range&#39;</span><span class="p">,</span> <span class="n">data_loader</span><span class="o">.</span><span class="n">dataset</span><span class="o">.</span><span class="n">time_range</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">&#39;Traffic data shape:&#39;</span><span class="p">,</span> <span class="n">data_loader</span><span class="o">.</span><span class="n">traffic_data</span><span class="o">.</span><span class="n">shape</span><span class="p">)</span>
<span class="c1"># The first dimension of data_loader.traffic_data is the length of time-sequence.</span>
<span class="c1"># The second dimension is the number of stations.</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">&#39;Time fitness:&#39;</span><span class="p">,</span> <span class="n">data_loader</span><span class="o">.</span><span class="n">dataset</span><span class="o">.</span><span class="n">time_fitness</span><span class="p">,</span> <span class="s1">&#39;minutes&#39;</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">&#39;Time sequence length:&#39;</span><span class="p">,</span> <span class="n">data_loader</span><span class="o">.</span><span class="n">traffic_data</span><span class="o">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">&#39;Number of stations:&#39;</span><span class="p">,</span> <span class="n">data_loader</span><span class="o">.</span><span class="n">traffic_data</span><span class="o">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">1</span><span class="p">])</span>
</pre></div>
</div>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">Data</span> <span class="n">time</span> <span class="nb">range</span> <span class="p">[</span><span class="s1">&#39;2013-07-01&#39;</span><span class="p">,</span> <span class="s1">&#39;2017-09-30&#39;</span><span class="p">]</span>
<span class="n">Traffic</span> <span class="n">data</span> <span class="n">shape</span><span class="p">:</span> <span class="p">(</span><span class="mi">3724</span><span class="p">,</span> <span class="mi">717</span><span class="p">)</span>
<span class="n">Time</span> <span class="n">fitness</span><span class="p">:</span> <span class="mi">60</span> <span class="n">minutes</span>
<span class="n">Time</span> <span class="n">sequence</span> <span class="n">length</span><span class="p">:</span> <span class="mi">3724</span>
<span class="n">Number</span> <span class="n">of</span> <span class="n">stations</span><span class="p">:</span> <span class="mi">717</span>
</pre></div>
</div>
<p>Visualize the distribution of the traffic data:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">matplotlib.pyplot</span> <span class="k">as</span> <span class="nn">plt</span>
<span class="n">plt</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">data_loader</span><span class="o">.</span><span class="n">traffic_data</span><span class="p">[:,</span> <span class="mi">0</span><span class="p">])</span>
<span class="n">plt</span><span class="o">.</span><span class="n">show</span><span class="p">()</span>
</pre></div>
</div>
<p><img alt="png" src="https://uctb.github.io/UCTB/sphinx/md_file/src/image/toturial_p1_dataplot.png" /></p>
</div>
<div class="section" id="build-your-own-datasets">
<h2>4.2. Build your own datasets<a class="headerlink" href="#build-your-own-datasets" title="Permalink to this headline">¶</a></h2>
<p>To make loader APIs compatible with your own data, you can store it in a <code class="docutils literal notranslate"><span class="pre">dict</span></code> variable with formats as follows.</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="c1"># Let&#39;s say ``my_dataset`` is your dataset.</span>
<span class="n">my_dataset</span> <span class="o">=</span> <span class="p">{</span>
    <span class="s2">&quot;TimeRange&quot;</span><span class="p">:</span> <span class="p">[</span><span class="s1">&#39;YYYY-MM-DD&#39;</span><span class="p">,</span> <span class="s1">&#39;YYYY-MM-DD&#39;</span><span class="p">],</span>
    <span class="s2">&quot;TimeFitness&quot;</span><span class="p">:</span> <span class="mi">60</span><span class="p">,</span> <span class="c1"># Minutes</span>
    
    <span class="s2">&quot;Node&quot;</span><span class="p">:</span> <span class="p">{</span>
        <span class="s2">&quot;TrafficNode&quot;</span><span class="p">:</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">,</span> <span class="c1"># With shape [time, num-of-node]</span>
        <span class="s2">&quot;TrafficMonthlyInteraction&quot;</span><span class="p">:</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">,</span> <span class="c1"># With shape [month, num-of-node. num-of-node]</span>
        <span class="s2">&quot;StationInfo&quot;</span><span class="p">:</span> <span class="nb">list</span> <span class="c1"># elements in it should be [id, build-time, lat, lng, name]</span>
        <span class="s2">&quot;POI&quot;</span><span class="p">:</span> <span class="p">[]</span>
    <span class="p">},</span>

    <span class="s2">&quot;Grid&quot;</span><span class="p">:</span> <span class="p">{</span>
        <span class="s2">&quot;TrafficGrid&quot;</span><span class="p">:</span> <span class="p">[],</span>
        <span class="s2">&quot;GridLatLng&quot;</span><span class="p">:</span> <span class="p">[],</span>
        <span class="s2">&quot;POI&quot;</span><span class="p">:</span> <span class="p">[]</span>
    <span class="p">},</span>

    <span class="s2">&quot;ExternalFeature&quot;</span><span class="p">:</span> <span class="p">{</span>
         <span class="s2">&quot;Weather&quot;</span><span class="p">:</span> <span class="p">[</span><span class="n">time</span><span class="p">,</span> <span class="n">weather</span><span class="o">-</span><span class="n">feature</span><span class="o">-</span><span class="n">dim</span><span class="p">]</span>
    <span class="p">}</span>
<span class="p">}</span>
</pre></div>
</div>
<p>And then save it with package <code class="docutils literal notranslate"><span class="pre">pickle</span></code> to a local path <code class="docutils literal notranslate"><span class="pre">pkl_file_name</span></code>.</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">pickle</span>
<span class="n">pkl_file_name</span> <span class="o">=</span> <span class="s1">&#39;./my_dataset.pkl&#39;</span>  
<span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="n">pkl_file_name</span><span class="p">,</span> <span class="s1">&#39;wb&#39;</span><span class="p">)</span> <span class="k">as</span> <span class="n">handle</span><span class="p">:</span>
    <span class="n">pickle</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span><span class="n">my_dataset</span><span class="p">,</span> <span class="n">handle</span><span class="p">,</span> <span class="n">protocol</span><span class="o">=</span><span class="n">pickle</span><span class="o">.</span><span class="n">HIGHEST_PROTOCOL</span><span class="p">)</span>
</pre></div>
</div>
<p>Finally, you can make uses of your dataset by UCTB’s loader APIs:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">data_loader</span> <span class="o">=</span> <span class="n">NodeTrafficLoader</span><span class="p">(</span><span class="n">dataset</span><span class="o">=</span><span class="n">pkl_file_name</span><span class="p">)</span>
</pre></div>
</div>
<p>For better understanding how to build dataset used in UCTB from the original dataset, here are two examples:</p>
<ol class="simple">
<li><p>Regional trajectory data for Xi’an and Chengdu.  <a class="reference external" href="./static/MakeDatasetDiDi.html">Guide</a></p></li>
<li><p><a class="reference external" href="https://github.com/didi/TrafficIndex">TTI</a> data for Chengdu, Jinan, Haikou, Shenzhen, Suzhou and Xi’an.  <a class="reference external" href="./static/MakeDatasetDiDi_TTI.html">Guide</a></p></li>
</ol>
<p>The above data comes from <a class="reference external" href="https://outreach.didichuxing.com/research/opendata/en/">https://gaia.didichuxing.com</a>, and we are especially grateful for the data provided by the DiDi Chuxing GAIA Initiative.</p>
</div>
<div class="section" id="use-build-in-models-from-uctb">
<h2>4.3. Use build-in models from UCTB<a class="headerlink" href="#use-build-in-models-from-uctb" title="Permalink to this headline">¶</a></h2>
<div class="section" id="use-single-temporal-feature-in-regression">
<h3>4.3.1. Use single temporal feature in regression<a class="headerlink" href="#use-single-temporal-feature-in-regression" title="Permalink to this headline">¶</a></h3>
<p>UCTB provides many classical and popular spatial-temporal predicting models. These models can be used to either predicting series for a single station or all stations. You can find the details in <a class="reference external" href="./static/current_supported_models.html"><code class="docutils literal notranslate"><span class="pre">UCTB.model</span></code></a>.</p>
<p>The following example shows how to use a <strong>Hidden Markov model (HMM)</strong> to handle a simple time series predicting a problem. We will try to predict the bike demands <code class="docutils literal notranslate"><span class="pre">test_y</span></code> of a fixed station <code class="docutils literal notranslate"><span class="pre">target_node</span></code> in New York City by checking back the historical demands in recent time slots <code class="docutils literal notranslate"><span class="pre">train_closeness</span></code>.</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
<span class="kn">import</span> <span class="nn">matplotlib.pyplot</span> <span class="k">as</span> <span class="nn">plt</span>

<span class="kn">from</span> <span class="nn">UCTB.model</span> <span class="kn">import</span> <span class="n">HMM</span>
<span class="kn">from</span> <span class="nn">UCTB.dataset</span> <span class="kn">import</span> <span class="n">NodeTrafficLoader</span>
<span class="kn">from</span> <span class="nn">UCTB.evaluation</span> <span class="kn">import</span> <span class="n">metric</span>

<span class="n">target_node</span> <span class="o">=</span> <span class="mi">233</span>
</pre></div>
</div>
<p>When initializing the loader, we use past <code class="docutils literal notranslate"><span class="pre">12</span></code> time slots (timesteps) of closeness as input, <code class="docutils literal notranslate"><span class="pre">1</span></code> timestep in the next as output and set the timesteps of other features <code class="docutils literal notranslate"><span class="pre">period_len</span></code>, <code class="docutils literal notranslate"><span class="pre">period_len</span></code> to zero.</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">data_loader</span> <span class="o">=</span> <span class="n">NodeTrafficLoader</span><span class="p">(</span><span class="n">data_range</span><span class="o">=</span><span class="mf">0.1</span><span class="p">,</span> <span class="n">dataset</span><span class="o">=</span><span class="s1">&#39;Bike&#39;</span><span class="p">,</span> <span class="n">city</span><span class="o">=</span><span class="s1">&#39;NYC&#39;</span><span class="p">,</span>
                                <span class="n">closeness_len</span><span class="o">=</span><span class="mi">12</span><span class="p">,</span> <span class="n">period_len</span><span class="o">=</span><span class="mi">0</span><span class="p">,</span> <span class="n">trend_len</span><span class="o">=</span><span class="mi">0</span><span class="p">,</span>
                                <span class="n">target_length</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">test_ratio</span><span class="o">=</span><span class="mf">0.2</span><span class="p">,</span> 
                                <span class="n">normalize</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">with_lm</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">with_tpe</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
</pre></div>
</div>
<p>The well-loaded data contain all <code class="docutils literal notranslate"><span class="pre">717</span></code> stations’ data. Therefore it is needed to specify the target station by <code class="docutils literal notranslate"><span class="pre">target_station</span></code>.</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="nb">print</span><span class="p">(</span><span class="n">data_loader</span><span class="o">.</span><span class="n">train_closeness</span><span class="o">.</span><span class="n">shape</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="n">data_loader</span><span class="o">.</span><span class="n">test_closeness</span><span class="o">.</span><span class="n">shape</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="n">data_loader</span><span class="o">.</span><span class="n">test_y</span><span class="o">.</span><span class="n">shape</span><span class="p">)</span>
</pre></div>
</div>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="p">(</span><span class="mi">2967</span><span class="p">,</span> <span class="mi">717</span><span class="p">,</span> <span class="mi">12</span><span class="p">,</span> <span class="mi">1</span><span class="p">)</span>
<span class="p">(</span><span class="mi">745</span><span class="p">,</span> <span class="mi">717</span><span class="p">,</span> <span class="mi">12</span><span class="p">,</span> <span class="mi">1</span><span class="p">)</span>
<span class="p">(</span><span class="mi">745</span><span class="p">,</span> <span class="mi">717</span><span class="p">,</span> <span class="mi">1</span><span class="p">)</span>
</pre></div>
</div>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">train_x</span><span class="p">,</span> <span class="n">test_x</span> <span class="o">=</span> <span class="n">data_loader</span><span class="o">.</span><span class="n">train_closeness</span><span class="p">[:,</span> <span class="n">target_node</span><span class="p">:</span><span class="n">target_node</span><span class="o">+</span><span class="mi">1</span><span class="p">,</span> <span class="o">-</span><span class="mi">1</span><span class="p">,</span> <span class="mi">0</span><span class="p">],</span> <span class="n">data_loader</span><span class="o">.</span><span class="n">test_closeness</span><span class="p">[:,</span> <span class="n">target_node</span><span class="p">,</span> <span class="p">:,</span> <span class="p">:]</span>
<span class="n">test_y</span> <span class="o">=</span> <span class="n">data_loader</span><span class="o">.</span><span class="n">test_y</span><span class="p">[:,</span> <span class="n">target_node</span><span class="p">,</span> <span class="mi">0</span><span class="p">]</span>
</pre></div>
</div>
<p>Inspect the shape of data. Here are the all we need for one-station prediction.</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="nb">print</span><span class="p">(</span><span class="n">train_x</span><span class="o">.</span><span class="n">shape</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="n">test_x</span><span class="o">.</span><span class="n">shape</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="n">test_y</span><span class="o">.</span><span class="n">shape</span><span class="p">)</span>
</pre></div>
</div>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">(</span><span class="mi">2967</span><span class="p">,</span> <span class="mi">1</span><span class="p">)</span>
<span class="p">(</span><span class="mi">745</span><span class="p">,</span> <span class="mi">12</span><span class="p">,</span> <span class="mi">1</span><span class="p">)</span>
<span class="p">(</span><span class="mi">745</span><span class="p">,)</span>
</pre></div>
</div>
<p>Build the HMM model.</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">model</span> <span class="o">=</span> <span class="n">HMM</span><span class="p">(</span><span class="n">num_components</span><span class="o">=</span><span class="mi">8</span><span class="p">,</span> <span class="n">n_iter</span><span class="o">=</span><span class="mi">50</span><span class="p">)</span>
</pre></div>
</div>
<p>Now, we can fit the model with the train dataset.</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">model</span><span class="o">.</span><span class="n">fit</span><span class="p">(</span><span class="n">x</span><span class="o">=</span><span class="n">train_x</span><span class="p">)</span>
</pre></div>
</div>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">Status</span><span class="p">:</span> <span class="n">converged</span>
</pre></div>
</div>
<p>When the model is converged, we make predictions on test data.</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">predictions</span> <span class="o">=</span> <span class="p">[]</span>
<span class="k">for</span> <span class="n">t</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">test_x</span><span class="o">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">0</span><span class="p">]):</span>
    <span class="n">p</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">squeeze</span><span class="p">(</span><span class="n">model</span><span class="o">.</span><span class="n">predict</span><span class="p">(</span><span class="n">x</span><span class="o">=</span><span class="n">test_x</span><span class="p">[</span><span class="n">t</span><span class="p">],</span> <span class="n">length</span><span class="o">=</span><span class="mi">1</span><span class="p">))</span>
    <span class="n">predictions</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">p</span><span class="p">)</span>
</pre></div>
</div>
<p>We can evaluate the performance of the model by build-in <code class="docutils literal notranslate"><span class="pre">UCTB.evaluation</span></code> APIs.</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">test_rmse</span> <span class="o">=</span> <span class="n">metric</span><span class="o">.</span><span class="n">rmse</span><span class="p">(</span><span class="n">predictions</span><span class="p">,</span> <span class="n">test_y</span><span class="p">,</span> <span class="n">threshold</span><span class="o">=</span><span class="mi">0</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="n">test_rmse</span><span class="p">)</span>
</pre></div>
</div>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="mf">3.76137200105079</span>
</pre></div>
</div>
</div>
<div class="section" id="use-multiple-temporal-features-in-regression">
<h3>4.3.2. Use multiple temporal features in regression<a class="headerlink" href="#use-multiple-temporal-features-in-regression" title="Permalink to this headline">¶</a></h3>
<p>In this case, let’s take more temporal knowledge related to <code class="docutils literal notranslate"><span class="pre">target_node</span></code> into account. We will concatenate factors including <code class="docutils literal notranslate"><span class="pre">closeness</span></code>, <code class="docutils literal notranslate"><span class="pre">period</span></code>, and <code class="docutils literal notranslate"><span class="pre">trend</span></code>, and use <strong>XGBoost</strong> as the predicting model.</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
<span class="kn">import</span> <span class="nn">matplotlib.pyplot</span> <span class="k">as</span> <span class="nn">plt</span>

<span class="kn">from</span> <span class="nn">UCTB.model</span> <span class="kn">import</span> <span class="n">XGBoost</span>
<span class="kn">from</span> <span class="nn">UCTB.dataset</span> <span class="kn">import</span> <span class="n">NodeTrafficLoader</span>
<span class="kn">from</span> <span class="nn">UCTB.evaluation</span> <span class="kn">import</span> <span class="n">metric</span>

<span class="n">target_node</span> <span class="o">=</span> <span class="mi">233</span>

<span class="n">data_loader</span> <span class="o">=</span> <span class="n">NodeTrafficLoader</span><span class="p">(</span><span class="n">data_range</span><span class="o">=</span><span class="mf">0.1</span><span class="p">,</span> <span class="n">dataset</span><span class="o">=</span><span class="s1">&#39;Bike&#39;</span><span class="p">,</span> <span class="n">city</span><span class="o">=</span><span class="s1">&#39;NYC&#39;</span><span class="p">,</span>
                                <span class="n">closeness_len</span><span class="o">=</span><span class="mi">6</span><span class="p">,</span> <span class="n">period_len</span><span class="o">=</span><span class="mi">7</span><span class="p">,</span> <span class="n">trend_len</span><span class="o">=</span><span class="mi">4</span><span class="p">,</span>
                                <span class="n">target_length</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">test_ratio</span><span class="o">=</span><span class="mf">0.2</span><span class="p">,</span> 
                                <span class="n">normalize</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">with_lm</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">with_tpe</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>

<span class="n">train_closeness</span> <span class="o">=</span> <span class="n">data_loader</span><span class="o">.</span><span class="n">train_closeness</span><span class="p">[:,</span> <span class="n">target_node</span><span class="p">,</span> <span class="p">:,</span> <span class="mi">0</span><span class="p">]</span>
<span class="n">train_period</span> <span class="o">=</span> <span class="n">data_loader</span><span class="o">.</span><span class="n">train_period</span><span class="p">[:,</span> <span class="n">target_nodze</span><span class="p">,</span> <span class="p">:,</span> <span class="mi">0</span><span class="p">]</span>
<span class="n">train_trend</span> <span class="o">=</span> <span class="n">data_loader</span><span class="o">.</span><span class="n">train_trend</span><span class="p">[:,</span> <span class="n">target_node</span><span class="p">,</span> <span class="p">:,</span> <span class="mi">0</span><span class="p">]</span>
<span class="n">train_y</span> <span class="o">=</span> <span class="n">data_loader</span><span class="o">.</span><span class="n">train_y</span><span class="p">[:,</span> <span class="n">target_node</span><span class="p">,</span> <span class="mi">0</span><span class="p">]</span>

<span class="n">test_closeness</span> <span class="o">=</span> <span class="n">data_loader</span><span class="o">.</span><span class="n">test_closeness</span><span class="p">[:,</span> <span class="n">target_node</span><span class="p">,</span> <span class="p">:,</span> <span class="mi">0</span><span class="p">]</span>
<span class="n">test_period</span> <span class="o">=</span> <span class="n">data_loader</span><span class="o">.</span><span class="n">test_period</span><span class="p">[:,</span> <span class="n">target_node</span><span class="p">,</span> <span class="p">:,</span> <span class="mi">0</span><span class="p">]</span>
<span class="n">test_trend</span> <span class="o">=</span> <span class="n">data_loader</span><span class="o">.</span><span class="n">test_trend</span><span class="p">[:,</span> <span class="n">target_node</span><span class="p">,</span> <span class="p">:,</span> <span class="mi">0</span><span class="p">]</span>
<span class="n">test_y</span> <span class="o">=</span> <span class="n">data_loader</span><span class="o">.</span><span class="n">test_y</span><span class="p">[:,</span> <span class="n">target_node</span><span class="p">,</span> <span class="mi">0</span><span class="p">]</span>

<span class="n">train_X</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">concatenate</span><span class="p">([</span><span class="n">train_closeness</span><span class="p">,</span> <span class="n">train_period</span><span class="p">,</span> <span class="n">train_trend</span><span class="p">],</span> <span class="n">axis</span><span class="o">=-</span><span class="mi">1</span><span class="p">)</span>
<span class="n">test_X</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">concatenate</span><span class="p">([</span><span class="n">test_closeness</span><span class="p">,</span> <span class="n">test_period</span><span class="p">,</span> <span class="n">test_trend</span><span class="p">],</span> <span class="n">axis</span><span class="o">=-</span><span class="mi">1</span><span class="p">)</span>

<span class="nb">print</span><span class="p">(</span><span class="n">train_X</span><span class="o">.</span><span class="n">shape</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="n">train_y</span><span class="o">.</span><span class="n">shape</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="n">test_X</span><span class="o">.</span><span class="n">shape</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="n">test_y</span><span class="o">.</span><span class="n">shape</span><span class="p">)</span>

<span class="n">model</span> <span class="o">=</span> <span class="n">XGBoost</span><span class="p">(</span><span class="n">n_estimators</span><span class="o">=</span><span class="mi">100</span><span class="p">,</span> <span class="n">max_depth</span><span class="o">=</span><span class="mi">3</span><span class="p">,</span> <span class="n">objective</span><span class="o">=</span><span class="s1">&#39;reg:linear&#39;</span><span class="p">)</span>
<span class="n">model</span><span class="o">.</span><span class="n">fit</span><span class="p">(</span><span class="n">train_X</span><span class="p">,</span> <span class="n">train_y</span><span class="p">)</span>
<span class="n">predictions</span> <span class="o">=</span> <span class="n">model</span><span class="o">.</span><span class="n">predict</span><span class="p">(</span><span class="n">test_X</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">&#39;Test RMSE&#39;</span><span class="p">,</span> <span class="n">metric</span><span class="o">.</span><span class="n">rmse</span><span class="p">(</span><span class="n">predictions</span><span class="p">,</span> <span class="n">test_y</span><span class="p">,</span> <span class="n">threshold</span><span class="o">=</span><span class="mi">0</span><span class="p">))</span>
</pre></div>
</div>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">(</span><span class="mi">2307</span><span class="p">,</span> <span class="mi">17</span><span class="p">)</span>
<span class="p">(</span><span class="mi">2307</span><span class="p">,)</span>
<span class="p">(</span><span class="mi">745</span><span class="p">,</span> <span class="mi">17</span><span class="p">)</span>
<span class="p">(</span><span class="mi">745</span><span class="p">,)</span>
<span class="n">Test</span> <span class="n">RMSE</span> <span class="mf">3.3267457</span>
</pre></div>
</div>
</div>
</div>
<div class="section" id="build-your-own-model-using-uctb">
<h2>4.4. Build your own model using UCTB<a class="headerlink" href="#build-your-own-model-using-uctb" title="Permalink to this headline">¶</a></h2>
<p>UCTB provides extendable APIs to build your own model. Currently, it can support the running of all the <code class="docutils literal notranslate"><span class="pre">1.x</span></code> version of <strong>Tensorflow-based</strong> models. In the following tutorial, we will show you how to takes the least efforts to implement a UCTB model.</p>
<p>Commonly, a new model needs to inherit <code class="docutils literal notranslate"><span class="pre">BaseModel</span></code> to acquire the features provided by UCTB, such as batch division, early stopping, etc. The necessary components for a subclass of <code class="docutils literal notranslate"><span class="pre">BaseModel</span></code> include:</p>
<ul class="simple">
<li><p><code class="docutils literal notranslate"><span class="pre">self.__init__()</span></code>. Define the model’s parameters related to the architecture. You should call the super class’s constructor at first.</p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">self.build()</span></code>. Build the architecture here. You should construct the graph at the beginning of this function and call the super class’s <code class="docutils literal notranslate"><span class="pre">build()</span></code> function at the end.</p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">self._input</span></code>. The <code class="docutils literal notranslate"><span class="pre">dict</span></code> used to record the acceptable inputs of the model, whose keys are the parameter names in <code class="docutils literal notranslate"><span class="pre">model.fit()</span></code> and <code class="docutils literal notranslate"><span class="pre">model.predict()</span></code> and values are the name of related tensors.</p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">self._output</span></code>. The <code class="docutils literal notranslate"><span class="pre">dict</span></code> used to record the outputs of the model. You should fill the required keys <code class="docutils literal notranslate"><span class="pre">prediction</span></code> and <code class="docutils literal notranslate"><span class="pre">loss</span></code> with the names of tensors in your case.</p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">self._op</span></code>. The <code class="docutils literal notranslate"><span class="pre">dict</span></code> used to define all the operations for the model. Basic usage for it is to record the <strong>training operation</strong>, for example, the minimizing loss operation of an optimizer. Use key <code class="docutils literal notranslate"><span class="pre">train_op</span></code> to record it.</p></li>
</ul>
<p>For more examples, you can refer to the implementations of build-in models in <a class="reference external" href="../UCTB.model.html#uctb-model-package"><code class="docutils literal notranslate"><span class="pre">UCTB.model</span></code></a>.</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">UCTB.model_unit</span> <span class="kn">import</span> <span class="n">BaseModel</span>

<span class="k">class</span> <span class="nc">MyModel</span><span class="p">(</span><span class="n">BaseModel</span><span class="p">):</span>
    <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span>
                 
                 <span class="n">code_version</span><span class="o">=</span><span class="s1">&#39;0&#39;</span><span class="p">,</span>
                 <span class="n">model_dir</span><span class="o">=</span><span class="s1">&#39;my_model&#39;</span><span class="p">,</span>
                 <span class="n">gpu_device</span><span class="o">=</span><span class="s1">&#39;0&#39;</span><span class="p">,</span>
                <span class="p">):</span>
        <span class="nb">super</span><span class="p">(</span><span class="n">MyModel</span><span class="p">,</span> <span class="bp">self</span><span class="p">)</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="n">code_version</span><span class="o">=</span><span class="n">code_version</span><span class="p">,</span> 
                                      <span class="n">model_dir</span><span class="o">=</span><span class="n">model_dir</span><span class="p">,</span> <span class="n">gpu_device</span><span class="o">=</span><span class="n">gpu_device</span><span class="p">)</span>
        <span class="o">...</span>
        
    <span class="k">def</span> <span class="nf">build</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">init_vars</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">max_to_keep</span><span class="o">=</span><span class="mi">5</span><span class="p">):</span>
        <span class="k">with</span> <span class="bp">self</span><span class="o">.</span><span class="n">_graph</span><span class="o">.</span><span class="n">as_default</span><span class="p">():</span>
            <span class="o">...</span>
            <span class="bp">self</span><span class="o">.</span><span class="n">_input</span><span class="p">[</span><span class="s1">&#39;inputs&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">inputs</span><span class="o">.</span><span class="n">name</span>
            <span class="bp">self</span><span class="o">.</span><span class="n">_input</span><span class="p">[</span><span class="s1">&#39;targets&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">targets</span><span class="o">.</span><span class="n">name</span>
            
            <span class="o">...</span>
            <span class="bp">self</span><span class="o">.</span><span class="n">_output</span><span class="p">[</span><span class="s1">&#39;prediction&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">predictions</span><span class="o">.</span><span class="n">name</span>
            <span class="bp">self</span><span class="o">.</span><span class="n">_output</span><span class="p">[</span><span class="s1">&#39;loss&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">loss</span><span class="o">.</span><span class="n">name</span>
            <span class="bp">self</span><span class="o">.</span><span class="n">_op</span><span class="p">[</span><span class="s1">&#39;train_op&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">train_op</span><span class="o">.</span><span class="n">name</span>
            
        <span class="nb">super</span><span class="p">(</span><span class="n">MyModel</span><span class="p">,</span> <span class="bp">self</span><span class="p">)</span><span class="o">.</span><span class="n">build</span><span class="p">(</span><span class="n">init_vars</span><span class="o">=</span><span class="n">init_vars</span><span class="p">,</span> <span class="n">max_to_keep</span><span class="o">=</span><span class="mi">5</span><span class="p">)</span> 
</pre></div>
</div>
<p>Next, in a concrete case, we will realize a <strong>Long short-term memory (LSTM)</strong> model to make the all-station prediction that accepts time series of <code class="docutils literal notranslate"><span class="pre">717</span></code> stations and predict the future of them as a whole.</p>
<p>For the mechanism of LSTM, you can refer to
<a class="reference external" href="https://www.researchgate.net/profile/Felix_Gers/publication/12292425_Learning_to_Forget_Continual_Prediction_with_LSTM/links/5759414608ae9a9c954e84c5/Learning-to-Forget-Continual-Prediction-with-LSTM.pdf">Gers, F. A., Schmidhuber, J., &amp; Cummins, F. (1999). Learning to forget: Continual prediction with LSTM</a>.</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
<span class="kn">import</span> <span class="nn">tensorflow</span> <span class="k">as</span> <span class="nn">tf</span>
<span class="kn">from</span> <span class="nn">UCTB.dataset</span> <span class="kn">import</span> <span class="n">NodeTrafficLoader</span>
<span class="kn">from</span> <span class="nn">UCTB.model_unit</span> <span class="kn">import</span> <span class="n">BaseModel</span>
<span class="kn">from</span> <span class="nn">UCTB.preprocess</span> <span class="kn">import</span> <span class="n">SplitData</span>
<span class="kn">from</span> <span class="nn">UCTB.evaluation</span> <span class="kn">import</span> <span class="n">metric</span>
</pre></div>
</div>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">LSTM</span><span class="p">(</span><span class="n">BaseModel</span><span class="p">):</span>
    <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span>
                 <span class="n">num_stations</span><span class="p">,</span> 
                 <span class="n">num_layers</span><span class="p">,</span> 
                 <span class="n">num_units</span><span class="p">,</span> 
                 <span class="n">input_steps</span><span class="p">,</span> 
                 <span class="n">input_dim</span><span class="p">,</span>
                 <span class="n">output_steps</span><span class="p">,</span>
                 <span class="n">output_dim</span><span class="p">,</span>
                 <span class="n">code_version</span><span class="o">=</span><span class="s1">&#39;0&#39;</span><span class="p">,</span>
                 <span class="n">model_dir</span><span class="o">=</span><span class="s1">&#39;my_lstm&#39;</span><span class="p">,</span>
                 <span class="n">gpu_device</span><span class="o">=</span><span class="s1">&#39;0&#39;</span><span class="p">):</span>
        <span class="nb">super</span><span class="p">(</span><span class="n">LSTM</span><span class="p">,</span> <span class="bp">self</span><span class="p">)</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="n">code_version</span><span class="o">=</span><span class="n">code_version</span><span class="p">,</span> 
                                   <span class="n">model_dir</span><span class="o">=</span><span class="n">model_dir</span><span class="p">,</span> <span class="n">gpu_device</span><span class="o">=</span><span class="n">gpu_device</span><span class="p">)</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">num_stations</span> <span class="o">=</span> <span class="n">num_stations</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">num_layers</span> <span class="o">=</span> <span class="n">num_layers</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">num_units</span> <span class="o">=</span> <span class="n">num_units</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">input_steps</span> <span class="o">=</span> <span class="n">input_steps</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">input_dim</span> <span class="o">=</span> <span class="n">input_dim</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">output_steps</span> <span class="o">=</span> <span class="n">output_steps</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">output_dim</span> <span class="o">=</span> <span class="n">output_dim</span>
        
    <span class="k">def</span> <span class="nf">build</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">init_vars</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">max_to_keep</span><span class="o">=</span><span class="mi">5</span><span class="p">):</span>
        <span class="k">with</span> <span class="bp">self</span><span class="o">.</span><span class="n">_graph</span><span class="o">.</span><span class="n">as_default</span><span class="p">():</span>
            <span class="n">inputs</span> <span class="o">=</span> <span class="n">tf</span><span class="o">.</span><span class="n">placeholder</span><span class="p">(</span><span class="n">tf</span><span class="o">.</span><span class="n">float32</span><span class="p">,</span> <span class="n">shape</span><span class="o">=</span><span class="p">(</span><span class="kc">None</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">num_stations</span><span class="p">,</span> 
                                                       <span class="bp">self</span><span class="o">.</span><span class="n">input_steps</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">input_dim</span><span class="p">))</span>
            <span class="n">targets</span> <span class="o">=</span> <span class="n">tf</span><span class="o">.</span><span class="n">placeholder</span><span class="p">(</span><span class="n">tf</span><span class="o">.</span><span class="n">float32</span><span class="p">,</span> <span class="n">shape</span><span class="o">=</span><span class="p">(</span><span class="kc">None</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">num_stations</span><span class="p">,</span>
                                                       <span class="bp">self</span><span class="o">.</span><span class="n">output_steps</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">output_dim</span><span class="p">))</span>
            <span class="c1"># record the inputs of the model</span>
            <span class="bp">self</span><span class="o">.</span><span class="n">_input</span><span class="p">[</span><span class="s1">&#39;inputs&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">inputs</span><span class="o">.</span><span class="n">name</span>
            <span class="bp">self</span><span class="o">.</span><span class="n">_input</span><span class="p">[</span><span class="s1">&#39;targets&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">targets</span><span class="o">.</span><span class="n">name</span>

            <span class="n">inputs</span> <span class="o">=</span> <span class="n">tf</span><span class="o">.</span><span class="n">reshape</span><span class="p">(</span><span class="n">inputs</span><span class="p">,</span> <span class="p">(</span><span class="o">-</span><span class="mi">1</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">input_steps</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">num_stations</span><span class="o">*</span><span class="bp">self</span><span class="o">.</span><span class="n">input_dim</span><span class="p">))</span>
            
            <span class="k">def</span> <span class="nf">get_a_cell</span><span class="p">(</span><span class="n">num_units</span><span class="p">):</span>
                <span class="n">lstm</span> <span class="o">=</span> <span class="n">tf</span><span class="o">.</span><span class="n">nn</span><span class="o">.</span><span class="n">rnn_cell</span><span class="o">.</span><span class="n">BasicLSTMCell</span><span class="p">(</span><span class="n">num_units</span><span class="p">,</span> <span class="n">state_is_tuple</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
                <span class="k">return</span> <span class="n">lstm</span>
            
            <span class="n">stacked_cells</span> <span class="o">=</span> <span class="n">tf</span><span class="o">.</span><span class="n">contrib</span><span class="o">.</span><span class="n">rnn</span><span class="o">.</span><span class="n">MultiRNNCell</span><span class="p">([</span><span class="n">get_a_cell</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">num_units</span><span class="p">)</span> <span class="k">for</span> <span class="n">_</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">num_layers</span><span class="p">)],</span> <span class="n">state_is_tuple</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
            <span class="n">outputs</span><span class="p">,</span> <span class="n">final_state</span> <span class="o">=</span> <span class="n">tf</span><span class="o">.</span><span class="n">nn</span><span class="o">.</span><span class="n">dynamic_rnn</span><span class="p">(</span><span class="n">stacked_cells</span><span class="p">,</span> <span class="n">inputs</span><span class="p">,</span> <span class="n">dtype</span><span class="o">=</span><span class="n">tf</span><span class="o">.</span><span class="n">float32</span><span class="p">)</span>
            
            <span class="n">stacked_outputs</span> <span class="o">=</span> <span class="n">tf</span><span class="o">.</span><span class="n">reshape</span><span class="p">(</span><span class="n">outputs</span><span class="p">,</span> <span class="n">shape</span><span class="o">=</span><span class="p">(</span><span class="o">-</span><span class="mi">1</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">num_units</span><span class="o">*</span><span class="bp">self</span><span class="o">.</span><span class="n">input_steps</span><span class="p">))</span>
            <span class="n">predictions</span> <span class="o">=</span> <span class="n">tf</span><span class="o">.</span><span class="n">layers</span><span class="o">.</span><span class="n">dense</span><span class="p">(</span><span class="n">stacked_outputs</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">output_steps</span><span class="o">*</span><span class="bp">self</span><span class="o">.</span><span class="n">num_stations</span><span class="o">*</span><span class="bp">self</span><span class="o">.</span><span class="n">output_dim</span><span class="p">)</span>
            <span class="n">predictions</span> <span class="o">=</span> <span class="n">tf</span><span class="o">.</span><span class="n">reshape</span><span class="p">(</span><span class="n">predictions</span><span class="p">,</span> <span class="n">shape</span><span class="o">=</span><span class="p">(</span><span class="o">-</span><span class="mi">1</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">num_stations</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">output_steps</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">output_dim</span><span class="p">))</span>
            
            <span class="n">loss</span> <span class="o">=</span> <span class="n">tf</span><span class="o">.</span><span class="n">sqrt</span><span class="p">(</span><span class="n">tf</span><span class="o">.</span><span class="n">reduce_mean</span><span class="p">(</span><span class="n">tf</span><span class="o">.</span><span class="n">square</span><span class="p">(</span><span class="n">predictions</span> <span class="o">-</span> <span class="n">targets</span><span class="p">)))</span>
            <span class="n">train_op</span> <span class="o">=</span> <span class="n">tf</span><span class="o">.</span><span class="n">train</span><span class="o">.</span><span class="n">AdamOptimizer</span><span class="p">()</span><span class="o">.</span><span class="n">minimize</span><span class="p">(</span><span class="n">loss</span><span class="p">)</span>
            
            <span class="c1"># record the outputs and the operation of the model</span>
            <span class="bp">self</span><span class="o">.</span><span class="n">_output</span><span class="p">[</span><span class="s1">&#39;prediction&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">predictions</span><span class="o">.</span><span class="n">name</span>
            <span class="bp">self</span><span class="o">.</span><span class="n">_output</span><span class="p">[</span><span class="s1">&#39;loss&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">loss</span><span class="o">.</span><span class="n">name</span>
            <span class="bp">self</span><span class="o">.</span><span class="n">_op</span><span class="p">[</span><span class="s1">&#39;train_op&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">train_op</span><span class="o">.</span><span class="n">name</span>
        
        <span class="c1"># must call super class&#39; function to build </span>
        <span class="nb">super</span><span class="p">(</span><span class="n">LSTM</span><span class="p">,</span> <span class="bp">self</span><span class="p">)</span><span class="o">.</span><span class="n">build</span><span class="p">(</span><span class="n">init_vars</span><span class="o">=</span><span class="n">init_vars</span><span class="p">,</span> <span class="n">max_to_keep</span><span class="o">=</span><span class="mi">5</span><span class="p">)</span> 
</pre></div>
</div>
<p>Load the dataset by loader and transform them into the formats your model accepts. If the loader APIs are not filled your demands, you can inherit loader and wrapper it according to your desires (see <a class="reference external" href="./quickstart.html">Quickstart</a> for more details).</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">data_loader</span> <span class="o">=</span> <span class="n">NodeTrafficLoader</span><span class="p">(</span><span class="n">data_range</span><span class="o">=</span><span class="mf">0.1</span><span class="p">,</span> <span class="n">dataset</span><span class="o">=</span><span class="s1">&#39;Bike&#39;</span><span class="p">,</span> <span class="n">city</span><span class="o">=</span><span class="s1">&#39;NYC&#39;</span><span class="p">,</span>
                                <span class="n">closeness_len</span><span class="o">=</span><span class="mi">6</span><span class="p">,</span> <span class="n">period_len</span><span class="o">=</span><span class="mi">0</span><span class="p">,</span> <span class="n">trend_len</span><span class="o">=</span><span class="mi">0</span><span class="p">,</span>
                                <span class="n">target_length</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">test_ratio</span><span class="o">=</span><span class="mf">0.2</span><span class="p">,</span> 
                                <span class="n">normalize</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">with_lm</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">with_tpe</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
<span class="n">train_y</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">expand_dims</span><span class="p">(</span><span class="n">data_loader</span><span class="o">.</span><span class="n">train_y</span><span class="p">,</span> <span class="n">axis</span><span class="o">=-</span><span class="mi">1</span><span class="p">)</span>
<span class="n">test_y</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">expand_dims</span><span class="p">(</span><span class="n">data_loader</span><span class="o">.</span><span class="n">test_y</span><span class="p">,</span> <span class="n">axis</span><span class="o">=-</span><span class="mi">1</span><span class="p">)</span>
</pre></div>
</div>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">model</span> <span class="o">=</span> <span class="n">LSTM</span><span class="p">(</span><span class="n">num_stations</span><span class="o">=</span><span class="n">data_loader</span><span class="o">.</span><span class="n">station_number</span><span class="p">,</span> 
             <span class="n">num_layers</span><span class="o">=</span><span class="mi">2</span><span class="p">,</span>
             <span class="n">num_units</span><span class="o">=</span><span class="mi">512</span><span class="p">,</span> 
             <span class="n">input_steps</span><span class="o">=</span><span class="mi">6</span><span class="p">,</span> 
             <span class="n">input_dim</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> 
             <span class="n">output_steps</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> 
             <span class="n">output_dim</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span>
</pre></div>
</div>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">model</span><span class="o">.</span><span class="n">build</span><span class="p">()</span>
<span class="nb">print</span><span class="p">(</span><span class="n">model</span><span class="o">.</span><span class="n">trainable_vars</span><span class="p">)</span>  <span class="c1"># count the trainble parameters</span>
</pre></div>
</div>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="mi">6821581</span>
</pre></div>
</div>
<p>Use your model to training and predicting. <code class="docutils literal notranslate"><span class="pre">model.fit()</span></code> method presets lots of useful functions, such as batch division and early stopping. Check them in <a class="reference external" href="../UCTB.model_unit.html#UCTB.model_unit.BaseModel.BaseModel.fit"><code class="docutils literal notranslate"><span class="pre">UCTB.model_unit.BaseModel.BaseModel.fit</span></code></a>.</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">model</span><span class="o">.</span><span class="n">fit</span><span class="p">(</span><span class="n">inputs</span><span class="o">=</span><span class="n">data_loader</span><span class="o">.</span><span class="n">train_closeness</span><span class="p">,</span>
          <span class="n">targets</span><span class="o">=</span><span class="n">train_y</span><span class="p">,</span>
          <span class="n">max_epoch</span><span class="o">=</span><span class="mi">10</span><span class="p">,</span>
          <span class="n">batch_size</span><span class="o">=</span><span class="mi">64</span><span class="p">,</span>
          <span class="n">sequence_length</span><span class="o">=</span><span class="n">data_loader</span><span class="o">.</span><span class="n">train_sequence_len</span><span class="p">,</span>
          <span class="n">validate_ratio</span><span class="o">=</span><span class="mf">0.1</span><span class="p">)</span>
</pre></div>
</div>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">No</span> <span class="n">model</span> <span class="n">found</span><span class="p">,</span> <span class="n">start</span> <span class="n">training</span>
<span class="n">Running</span> <span class="n">Operation</span> <span class="p">(</span><span class="s1">&#39;train_op&#39;</span><span class="p">,)</span>
<span class="n">Epoch</span> <span class="mi">0</span><span class="p">:</span> <span class="n">train_loss</span> <span class="mf">0.016053785</span> <span class="n">val_loss</span> <span class="mf">0.01606118</span>
<span class="n">Epoch</span> <span class="mi">1</span><span class="p">:</span> <span class="n">train_loss</span> <span class="mf">0.015499311</span> <span class="n">val_loss</span> <span class="mf">0.015820855</span>
<span class="n">Epoch</span> <span class="mi">2</span><span class="p">:</span> <span class="n">train_loss</span> <span class="mf">0.015298592</span> <span class="n">val_loss</span> <span class="mf">0.015657894</span>
<span class="n">Epoch</span> <span class="mi">3</span><span class="p">:</span> <span class="n">train_loss</span> <span class="mf">0.015163456</span> <span class="n">val_loss</span> <span class="mf">0.015559187</span>
<span class="n">Epoch</span> <span class="mi">4</span><span class="p">:</span> <span class="n">train_loss</span> <span class="mf">0.015066812</span> <span class="n">val_loss</span> <span class="mf">0.015342651</span>
<span class="n">Epoch</span> <span class="mi">5</span><span class="p">:</span> <span class="n">train_loss</span> <span class="mf">0.015016247</span> <span class="n">val_loss</span> <span class="mf">0.015287879</span>
<span class="n">Epoch</span> <span class="mi">6</span><span class="p">:</span> <span class="n">train_loss</span> <span class="mf">0.014899823</span> <span class="n">val_loss</span> <span class="mf">0.015249459</span>
<span class="n">Epoch</span> <span class="mi">7</span><span class="p">:</span> <span class="n">train_loss</span> <span class="mf">0.014773054</span> <span class="n">val_loss</span> <span class="mf">0.015098239</span>
<span class="n">Epoch</span> <span class="mi">8</span><span class="p">:</span> <span class="n">train_loss</span> <span class="mf">0.014655286</span> <span class="n">val_loss</span> <span class="mf">0.015097916</span>
<span class="n">Epoch</span> <span class="mi">9</span><span class="p">:</span> <span class="n">train_loss</span> <span class="mf">0.014558283</span> <span class="n">val_loss</span> <span class="mf">0.015108417</span>
</pre></div>
</div>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">predictions</span> <span class="o">=</span> <span class="n">model</span><span class="o">.</span><span class="n">predict</span><span class="p">(</span><span class="n">inputs</span><span class="o">=</span><span class="n">data_loader</span><span class="o">.</span><span class="n">test_closeness</span><span class="p">,</span> 
                            <span class="n">sequence_length</span><span class="o">=</span><span class="n">data_loader</span><span class="o">.</span><span class="n">test_sequence_len</span><span class="p">)</span>
</pre></div>
</div>
<p>Reverse the normalization by <code class="docutils literal notranslate"><span class="pre">data_loader</span></code> and evaluate the results:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">predictions</span> <span class="o">=</span> <span class="n">data_loader</span><span class="o">.</span><span class="n">normalizer</span><span class="o">.</span><span class="n">min_max_denormal</span><span class="p">(</span><span class="n">predictions</span><span class="p">[</span><span class="s1">&#39;prediction&#39;</span><span class="p">])</span>
<span class="n">targets</span> <span class="o">=</span> <span class="n">data_loader</span><span class="o">.</span><span class="n">normalizer</span><span class="o">.</span><span class="n">min_max_denormal</span><span class="p">(</span><span class="n">test_y</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">&#39;Test result&#39;</span><span class="p">,</span> <span class="n">metric</span><span class="o">.</span><span class="n">rmse</span><span class="p">(</span><span class="n">prediction</span><span class="o">=</span><span class="n">predictions</span><span class="p">,</span> <span class="n">target</span><span class="o">=</span><span class="n">targets</span><span class="p">,</span> <span class="n">threshold</span><span class="o">=</span><span class="mi">0</span><span class="p">))</span>
</pre></div>
</div>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">Test</span> <span class="n">result</span> <span class="mf">2.9765626570592545</span>
</pre></div>
</div>
<p>Since we only use a short period of the dataset (<code class="docutils literal notranslate"><span class="pre">data_range=0.1</span></code>) in this toy example, the result looks good compared with the <a class="reference external" href="./all_results.html#results-on-bike">experiment</a>. You can also take a try to test the completed dataset on your model.</p>
</div>
<div class="section" id="build-your-own-graph-with-stmeta">
<h2>4.5. Build your own graph with STMeta<a class="headerlink" href="#build-your-own-graph-with-stmeta" title="Permalink to this headline">¶</a></h2>
<p>Next, we will use the Top-K graph as an example to illustrate how to build customized graphs in UCTB. All of the code in this section can be found <a class="reference external" href="https://anonymous.4open.science/r/561305b5-e65e-46c6-9371-ae76b85109ee/Experiments/CustomizedDemo/">here</a>.</p>
<div class="section" id="top-k-graph">
<h3>4.5.1. Top-K graph<a class="headerlink" href="#top-k-graph" title="Permalink to this headline">¶</a></h3>
<p>First of all, the customized graphs used in this section is called Top-K graph. We construct the corresponding adjacent graph by marking the point pair that consist of each point and its nearest K points as 1, and the others are marked as 0. Then, we use the adjacent graph to generate the laplacian matrix for input. The hyperparameter K is designed via ad-hoc heuristics. In this demonstration, we chose 23 as the value of K.</p>
</div>
<div class="section" id="realize-topk-graph-analysis-module">
<h3>4.5.2. Realize TopK graph analysis module<a class="headerlink" href="#realize-topk-graph-analysis-module" title="Permalink to this headline">¶</a></h3>
<p>To adopt customized graphs (<em><strong>e.g.,</strong></em> Top-K) in UCTB, you should first build your own analysis class by inheriting <code class="docutils literal notranslate"><span class="pre">UCTB.preprocess.GraphGenerator</span> <span class="pre">class</span></code>.</p>
<p>It is worth noting that the ultimate goal is to generate the member variables: <code class="docutils literal notranslate"><span class="pre">self.LM</span></code> and <code class="docutils literal notranslate"><span class="pre">self.AM</span></code>, which is the input matrix of the graph. In this phase, we need to make the corresponding analytical implementation according to the type of the custom graph passed in.</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="c1"># &quot;UCTB/preprocess/topKGraph.py&quot;</span>
<span class="kn">import</span> <span class="nn">heapq</span>
<span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
<span class="kn">from</span> <span class="nn">UCTB.preprocess.GraphGenerator</span> <span class="kn">import</span> <span class="n">GraphGenerator</span>

<span class="c1"># Define the class: topKGraph</span>
<span class="k">class</span> <span class="nc">topKGraph</span><span class="p">(</span><span class="n">GraphGenerator</span><span class="p">):</span>  <span class="c1"># Init NodeTrafficLoader</span>

   <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span><span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>

       <span class="nb">super</span><span class="p">(</span><span class="n">topKGraph</span><span class="p">,</span> <span class="bp">self</span><span class="p">)</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
       
       <span class="k">for</span> <span class="n">graph_name</span> <span class="ow">in</span> <span class="n">kwargs</span><span class="p">[</span><span class="s1">&#39;graph&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s1">&#39;-&#39;</span><span class="p">):</span>

<span class="c1"># As the basic graph is implemented in GraphGenerator, you only need to implement your own graph function instead of the existing one.</span>
           <span class="k">if</span> <span class="n">graph_name</span><span class="o">.</span><span class="n">lower</span><span class="p">()</span> <span class="o">==</span> <span class="s1">&#39;topk&#39;</span><span class="p">:</span>
               <span class="n">lat_lng_list</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([[</span><span class="nb">float</span><span class="p">(</span><span class="n">e1</span><span class="p">)</span> <span class="k">for</span> <span class="n">e1</span> <span class="ow">in</span> <span class="n">e</span><span class="p">[</span><span class="mi">2</span><span class="p">:</span><span class="mi">4</span><span class="p">]]</span>
                                        <span class="k">for</span> <span class="n">e</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">dataset</span><span class="o">.</span><span class="n">node_station_info</span><span class="p">])</span>
               <span class="c1"># Handling</span>
               <span class="n">AM</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">neighbour_adjacent</span><span class="p">(</span><span class="n">lat_lng_list</span><span class="p">[</span><span class="bp">self</span><span class="o">.</span><span class="n">traffic_data_index</span><span class="p">],</span>
                                       <span class="n">threshold</span><span class="o">=</span><span class="nb">int</span><span class="p">(</span><span class="n">kwargs</span><span class="p">[</span><span class="s1">&#39;threshold_neighbour&#39;</span><span class="p">]))</span>
               <span class="n">LM</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">adjacent_to_laplacian</span><span class="p">(</span><span class="n">AM</span><span class="p">)</span>

               <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">AM</span><span class="o">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">==</span> <span class="mi">0</span><span class="p">:</span>  <span class="c1"># Make AM</span>
                   <span class="bp">self</span><span class="o">.</span><span class="n">AM</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([</span><span class="n">AM</span><span class="p">],</span> <span class="n">dtype</span><span class="o">=</span><span class="n">np</span><span class="o">.</span><span class="n">float32</span><span class="p">)</span>
               <span class="k">else</span><span class="p">:</span>
                   <span class="bp">self</span><span class="o">.</span><span class="n">AM</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">vstack</span><span class="p">((</span><span class="bp">self</span><span class="o">.</span><span class="n">AM</span><span class="p">,</span> <span class="p">(</span><span class="n">AM</span><span class="p">[</span><span class="n">np</span><span class="o">.</span><span class="n">newaxis</span><span class="p">,</span> <span class="p">:])))</span>

               <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">LM</span><span class="o">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">==</span> <span class="mi">0</span><span class="p">:</span>  <span class="c1"># Make LM</span>
                   <span class="bp">self</span><span class="o">.</span><span class="n">LM</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([</span><span class="n">LM</span><span class="p">],</span> <span class="n">dtype</span><span class="o">=</span><span class="n">np</span><span class="o">.</span><span class="n">float32</span><span class="p">)</span>
               <span class="k">else</span><span class="p">:</span>
                   <span class="bp">self</span><span class="o">.</span><span class="n">LM</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">vstack</span><span class="p">((</span><span class="bp">self</span><span class="o">.</span><span class="n">LM</span><span class="p">,</span> <span class="p">(</span><span class="n">LM</span><span class="p">[</span><span class="n">np</span><span class="o">.</span><span class="n">newaxis</span><span class="p">,</span> <span class="p">:])))</span>

<span class="c1"># Implement the details of building the Top-K graph.</span>
   <span class="k">def</span> <span class="nf">neighbour_adjacent</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">lat_lng_list</span><span class="p">,</span> <span class="n">threshold</span><span class="p">):</span>
       <span class="n">adjacent_matrix</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">zeros</span><span class="p">([</span><span class="nb">len</span><span class="p">(</span><span class="n">lat_lng_list</span><span class="p">),</span> <span class="nb">len</span><span class="p">(</span><span class="n">lat_lng_list</span><span class="p">)])</span>
       <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">lat_lng_list</span><span class="p">)):</span>
           <span class="k">for</span> <span class="n">j</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">lat_lng_list</span><span class="p">)):</span>
               <span class="n">adjacent_matrix</span><span class="p">[</span><span class="n">i</span><span class="p">][</span><span class="n">j</span><span class="p">]</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">haversine</span><span class="p">(</span>
                   <span class="n">lat_lng_list</span><span class="p">[</span><span class="n">i</span><span class="p">][</span><span class="mi">0</span><span class="p">],</span> <span class="n">lat_lng_list</span><span class="p">[</span><span class="n">i</span><span class="p">][</span><span class="mi">1</span><span class="p">],</span> <span class="n">lat_lng_list</span><span class="p">[</span><span class="n">j</span><span class="p">][</span><span class="mi">0</span><span class="p">],</span> <span class="n">lat_lng_list</span><span class="p">[</span><span class="n">j</span><span class="p">][</span><span class="mi">1</span><span class="p">])</span>
       <span class="n">dis_matrix</span> <span class="o">=</span> <span class="n">adjacent_matrix</span><span class="o">.</span><span class="n">astype</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">float32</span><span class="p">)</span>

       <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">dis_matrix</span><span class="p">)):</span>
           <span class="n">ind</span> <span class="o">=</span> <span class="n">heapq</span><span class="o">.</span><span class="n">nlargest</span><span class="p">(</span><span class="n">threshold</span><span class="p">,</span> <span class="nb">range</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">dis_matrix</span><span class="p">[</span><span class="n">i</span><span class="p">])),</span> <span class="n">dis_matrix</span><span class="p">[</span><span class="n">i</span><span class="p">]</span><span class="o">.</span><span class="n">take</span><span class="p">)</span>
           <span class="n">dis_matrix</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([</span><span class="mi">0</span> <span class="k">for</span> <span class="n">_</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">dis_matrix</span><span class="p">[</span><span class="n">i</span><span class="p">]))])</span>
           <span class="n">dis_matrix</span><span class="p">[</span><span class="n">i</span><span class="p">][</span><span class="n">ind</span><span class="p">]</span> <span class="o">=</span> <span class="mi">1</span>
       <span class="n">adjacent_matrix</span> <span class="o">=</span> <span class="p">(</span><span class="n">adjacent_matrix</span> <span class="o">==</span> <span class="mi">1</span><span class="p">)</span><span class="o">.</span><span class="n">astype</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">float32</span><span class="p">)</span>
       <span class="k">return</span> <span class="n">adjacent_matrix</span>
</pre></div>
</div>
</div>
<div class="section" id="redefine-the-call-statement-of-the-above-class">
<h3>4.5.3. Redefine the call statement of the above class<a class="headerlink" href="#redefine-the-call-statement-of-the-above-class" title="Permalink to this headline">¶</a></h3>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="c1"># &quot;UCTB/Experiments/CustomizedDemo/STMeta_Obj_topk.py&quot;</span>

<span class="c1"># Import the Class: topKGraph</span>
<span class="kn">from</span> <span class="nn">topKGraph</span> <span class="kn">import</span> <span class="n">topKGraph</span>
<span class="c1"># Call topKGraph to initialize and generate AM and LM</span>
<span class="n">graphBuilder</span> <span class="o">=</span> <span class="n">topKGraph</span><span class="p">(</span><span class="n">graph</span><span class="o">=</span><span class="n">args</span><span class="p">[</span><span class="s1">&#39;graph&#39;</span><span class="p">],</span>
                         <span class="n">data_loader</span><span class="o">=</span><span class="n">data_loader</span><span class="p">,</span>
                         <span class="n">threshold_distance</span><span class="o">=</span><span class="n">args</span><span class="p">[</span><span class="s1">&#39;threshold_distance&#39;</span><span class="p">],</span>
                         <span class="n">threshold_correlation</span><span class="o">=</span><span class="n">args</span><span class="p">[</span><span class="s1">&#39;threshold_correlation&#39;</span><span class="p">],</span>
                         <span class="n">threshold_interaction</span><span class="o">=</span><span class="n">args</span><span class="p">[</span><span class="s1">&#39;threshold_interaction&#39;</span><span class="p">],</span>
                         <span class="n">threshold_neighbour</span><span class="o">=</span><span class="n">args</span><span class="p">[</span><span class="s1">&#39;threshold_neighbour&#39;</span><span class="p">])</span>
<span class="c1"># ......</span>
</pre></div>
</div>
</div>
<div class="section" id="modify-the-function-call-location">
<h3>4.5.4. Modify the function call location<a class="headerlink" href="#modify-the-function-call-location" title="Permalink to this headline">¶</a></h3>
<p>Add the new graph name when fitting model and then execute it for experiments. <a class="reference external" href="https://github.com/uctb/UCTB/blob/master/Experiments/CustomizedDemo/Runner_topk.py">code</a></p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">os</span><span class="o">.</span><span class="n">system</span><span class="p">(</span><span class="s1">&#39;python STMeta_Obj_topk.py -m STMeta_v1.model.yml -d metro_shanghai.data.yml &#39;</span>
          <span class="s1">&#39;-p graph:Distance-Correlation-Line-TopK,MergeIndex:12&#39;</span><span class="p">)</span>
</pre></div>
</div>
<p>We conduct experiments on <code class="docutils literal notranslate"><span class="pre">Metro_Shanghai</span></code> dataset and use the <a class="reference external" href="https://uctb.github.io/UCTB/md_file/all_results.html#stmeta-version">STMeta_V1</a> to model both “Distance-Correlation-Line” graph and “Distance-Correlation-Line-TopK” and the results are following:</p>
<table border="1" class="docutils">
<thead>
<tr>
<th align="center"><strong>Metro: Shanghai</strong></th>
<th align="center">Graph</th>
<th align="center">Test-RMSE</th>
</tr>
</thead>
<tbody>
<tr>
<td align="center">STMeta_V1</td>
<td align="center">Distance-Correlation-Line</td>
<td align="center">153.17</td>
</tr>
<tr>
<td align="center">STMeta_V1</td>
<td align="center">Distance-Correlation-Line-TopK</td>
<td align="center">140.82</td>
</tr>
</tbody>
</table><p>The results show that the performance of STMeta_V1 with the graph “Distance-Correlation-Line-TopK” is better than “Distance-Correlation-Line” model and the RMSE is reduced by about 12.4%, which validates the effectiveness of the topk graph for spatiotemporal modeling STMeta algorithm.</p>
<hr class="docutils" />
<p><u><a class="reference external" href="../index.html">Back To HomePage</a></u></p>
</div>
</div>
</div>


           </div>
          </div>
          <footer>
  
    <div class="rst-footer-buttons" role="navigation" aria-label="footer navigation">
      
        <a href="../APIReference.html" class="btn btn-neutral float-right" title="5. API Reference" accesskey="n">Next →</a>
      
      
        <a href="quickstart.html" class="btn btn-neutral" title="3. Quick start" accesskey="p">← Previous</a>
      
    </div>
  

  <hr/>

  <div role="contentinfo">
    <p>
        &copy; Copyright 2019, UCTB group.

    </p>
  </div>
  Built with <a href="http://sphinx-doc.org/">Sphinx</a>. 

</footer>

        </div>
      </div>

    </section>

  </div>
  


  

    <script type="text/javascript">
        var DOCUMENTATION_OPTIONS = {
            URL_ROOT:'../',
            VERSION:'',
            COLLAPSE_INDEX:false,
            FILE_SUFFIX:'.html',
            HAS_SOURCE: 'true'
        };
    </script>
      <script type="text/javascript" src="../_static/jquery.js"></script>
      <script type="text/javascript" src="../_static/underscore.js"></script>
      <script type="text/javascript" src="../_static/doctools.js"></script>
      <script type="text/javascript" src="../_static/language_data.js"></script>

  

  
  
    <script type="text/javascript" src="../_static/js/theme.js"></script>
  

  
  
  <script type="text/javascript">
      jQuery(function () {
          SphinxRtdTheme.StickyNav.enable();
      });
  </script>
   

</body>
</html>